Apparatus and methods for determining and processing medical outcomes

ABSTRACT

A data processing system is provided for determining clinical outcomes of medical data gathered by the system. The system can allow a person to define a medical study and can then administer the medical study and can collect and analyze data from potentially geographically diverse doctors, patients and other people associated with a study. Users enter sets of medical information. The system can analyze the medical data according to any number of clinical algorithms that may be custom defined and edited before and during the study. The system conditionally outputs the clinical outcome to the user. The clinical outcome can be used for treatment of patients participating in the study immediately after the data is input and analyzed. The medical outcomes can indicate such things as performance comparisons, composite outcomes, and risk stratification and assessments for such things as treatments, drugs, illnesses, doctors, patients and physician groups.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application that claims, under 35U.S.C. §120, priority to and the benefits of:

1. U.S. application Ser. No. 11/871,006, filed Oct. 11, 2007, which is acontinuation of U.S. application Ser. No. 09/330,384, filed Jun. 11,1999, which is a nonprovisional application claiming priority under 35U.S.C. § 119 to both U.S. Provisional Application No. 60/089,144 andU.S. Provisional Application No. 60/089,183 both filed Jun. 12, 1998;and

2. U.S. application Ser. No. 09/330,384, filed Jun. 11, 1999, which is anonprovisional application claiming priority under 35 U.S.C. § 119 toboth U.S. Provisional Application No. 60/089,144 and U.S. ProvisionalApplication No. 60/089,183 both filed Jun. 12, 1998.

This application is a nonprovisional application that claims, under 35U.S.C. § 119, priority to and the benefits of:

U.S. Provisional Application No. 60/089,144 filed Jun. 12, 1998; and

U.S. Provisional Application No. 60/089,183 filed Jun. 12, 1998

The disclosure of each prior application is incorporated herein byreference.

BACKGROUND

Medical outcomes research is routinely performed and often mandatedwithin the medical community. The goal of such research is to evaluatevarious factors which effect the health, well-being and medicaltreatment for people in society. Medical outcomes research can exist inmany forms. By way of example, such research can include the evaluationof drugs, ailments, diseases, doctors, treatment techniques, and soforth in relation to various factors such as recovery rates, costs,incidences of sickness, side effects, patient mortality and/or otherfactors. Specific clinical or medical studies are often used to conductmedical outcomes research, as a way of gathering data related to thespecific medical topic being researched.

Generally, to conduct a medical study on a specific medical topic,medical professionals such as doctors and other allied healthprofessionals, and sometimes selected patients involved in the study,provide medical information related to the topic of the study which isbeing researched. The medical information gathered is often general innature and may be obtained, for example, by reviewing patient recordsmaintained in a hospital. Due to various reasons, information that canbe obtained may be limited to only relevant factors within the patientfiles, and often excludes extraneous information that is not consideredrelated to the medical study. Once the medical information from thesevarious sources is gathered, the information is reviewed by a person incharge of conducting the study. The reviewer collates the informationinto various categories of data, which are collectively called medicaloutcomes for the study. Typically, the medical outcomes provide anindication, score or other criteria, based on the data collected in thestudy which may indicate, determine or evaluate the effectiveness orperformance of a doctor, drug, or medical treatment as related to thetreatment of a disease, sickness or other malady.

As an example, a drug manufacturer may sponsor a medical outcomes studyto determine the effectiveness of a new drug recently released into usefor the medical treatment of a specific ailment. The drug manufacturermay commission a medical research institute or other entity to researchhow many prescriptions of the drug have been filled, and of those filledprescriptions, how many of the patients who took the drug havesuccessfully recovered from the disease. This information may begathered over a long period of time across a wide range of thepopulation. Once the information has been collected, the entityconducting the study can analyze the data that was collected todetermine recovery rates for patients who took the drug. As a specificexample, if ten thousand prescriptions of the drug were filled over athree year period, and seven thousand patients who took the drugexperienced positive results from the drug treatment, the clinicaloutcome produced as a result of the research may indicate that the drughas a seventy percent effectiveness rating. The researching organizationcan then publish this clinical outcome in a report to inform the medicalpopulation and the public at large about the past performance of thedrug in relation to its effectiveness in treating a disease.

More detailed information on prior art techniques for conducting medicaloutcome studies and research is available in the text entitled “MedicalOutcomes & Guidelines Sourcebook”, published in 1998 in the UnitedStates by Faulkner & Gray's Healthcare Information Center, Eleven PennPlaza, New York, N.Y., the contents of which is hereby incorporated byreference in its entirety.

SUMMARY

Prior art techniques and methodologies for conducting medical outcomesresearch are fraught with deficiencies. Embodiments of the presentinvention are based, in part, on the recognition of these deficienciesand provide solutions to advance the state-of-the-art in medicaloutcomes research. A unique system and methodologies are disclosedherein for implementing, tracking, and processing medical outcomesresearch studies to produce clinical outcomes which result in a betterunderstanding of medical and health care knowledge related to thetreatment of sicknesses, illness, and disease and the treatment andperformance of doctors, treatment techniques (e.g., surgicaltechniques), drugs and other related medical research interests.

Prior art medical study implementation techniques are largely general innature with respect to the level of detail collected for the study.Generally, implementation of a study involves collecting data on formswhich are filled out by doctors or medical directors of physiciansgroups. For example, in a drug study, a form may request informationabout how many prescriptions of a particular drug have been given overthe past six months, and of those written, how many have resulted inpositive patient outcomes. This information may be gathered over timefrom a large percentage of the medical population that prescribed thedrug. The data may be collected over the course of many months or years,and once complete, the data is analyzed, possibly with the assistance ofa computer or database, and a medical outcome is determined that gives afairly accurate representation of the effectiveness of the drug attreating a particular illness. While such implementations of studies mayseem quite complete, vast amounts of data remain unrecorded or, ifrecorded, remain unanalyzed in the study research, and are thus notconsidered in the study results or outcomes. For instance, detailedhistories of every patient who took the drug are not analyzed in a largestudy due to the amount of time that would be required for suchanalysis. Using prior art medical study implementation techniques, itwould be unrealistic to consider all information gained from everypatient in the study.

Moreover, doctors themselves could provide a potentially large amount ofinformation concerning the actual use of a drug being studied. However,prior art study techniques rarely account for a detailed analysis of thebackground and actions of every doctor who administers the drug in thestudy. Due the lack of detailed analysis of data in such prior artstudies, the studies results may seem to accurately reflect theeffectiveness of the drug, but there may be missing information makingthe results somewhat less accurate than if all data were to beconsidered in determining a particular outcome.

Another problem with prior art medical study implementation techniquesis that the study results are made available long after the actualtreatment of most patients that participate in the study. Thus, if aprior art study technique does gather data at the patient level, patientinput is not used for study results that can then be used to treat thosepatients at that moment. More likely, the patient may have either fullyrecovered, or worse, may have died, before the results of the study canbe used to benefit the patient population at large. In other words,prior art study techniques gather data over long periods of time, andonce the data is gathered, at the end of the study, it is tabulated anda medical outcome is produced. The medical outcome may be beneficial atthat time, but has little effect or benefit on the treatment of thoseindividuals who partook in the study. Essentially, prior art studytechniques are retrospective in that they produce information thatbenefits future patients, but offers little benefit to the patients,researchers, and doctors involved during the study.

Conversely, the present invention provides a system to implement andconduct medical studies to produce medical outcomes that can be used totreat patients that are currently participating in the study. That is,clinical outcome data produced from the system of the invention can begenerated at any time, and in real-time, during the study. The medicaloutcome data is thus available for treatment of patients and can takeinto account all data entered by a doctor, patient, or other participantin the study, no matter where that person is geographically located.Thus, clinical outcomes produced from the system of the inventionreflect the most up-to-date progress and results of the study.

Moreover, the granularity at which data is collected in studiesadministered by the clinical outcome system of the invention allows manymore factors (than prior art systems) to be analyzed when generatingmedical outcome data. Data is collected on a patient by patient anddoctor by doctor level, which allows cross-correlations ofcharacteristics of medical data that were never before possible prior tothe system of this invention.

More specifically, the present invention provides an embodimentcomprising a clinical outcome system in the form of a digital dataprocessing system for determining clinical outcomes of medical data. Thedigital data processing system includes an input mechanism receivingsets of medical information. Each set has characteristics relating to aspecific medical study and the characteristics of each set also can haveassociated values. A storage mechanism is provided and is coupled to theinput mechanism. The storage mechanism receives and maintains the setsof medical information. A processor is provided and is coupled to thestorage mechanism. The processor selects a first characteristic commonto at least two sets of medical information. The processor processes allvalues of the first characteristic according to a clinical algorithm todetermine a clinical outcome of the sets of medical information for thespecific medical study based upon the selected first characteristic. Anoutput mechanism is also provided and is coupled to the processor toreceive the clinical outcome of the sets of medical information and tooutput the clinical outcome to a user of the digital data processingsystem. The system allows use of the clinical outcome during the studybased upon input received from the input mechanism. The clinical outcomemay indicate, for example, a performance of a medical treatment, adoctor, and/or a drug in comparison to other medical treatments,doctors, and drugs and this indication may be used during progression ofthe study to treat patients, for example.

Also in accordance with the invention, the sets of medical informationcan contain characteristics related to the specific medical study andcan include data related to a patient, a drug, an ailment, a doctor anda treatment technique. In such a configuration, the clinical outcome isdetermined based upon the selected first characteristic and indicates astatistical result derived from the clinical algorithm for at least oneof the patient, drug, ailment, treatment or doctor in relation toanother patient, drug, ailment, treatment and/or another doctor.

The processor in the clinical outcome system can also select a secondcharacteristic common to the sets of medical information. The processorcan process, using the clinical algorithm, the second characteristic incombination with the first characteristic to determine across-correlation between the first characteristic and the secondcharacteristic which is included in the clinical outcome for the sets ofmedical information related to the specific medical study. This allowsthe invention to cross-correlate any information in the medicalinformation to provide very detailed research results in the clinicaloutcome data.

For example, the first characteristic may be an identity of a doctor. Insuch a configuration, the clinical outcome can provide an indication ofthe performance of the doctor for a specific drug, a specific patient,and/or a specific ailment in comparison to other doctors. In anotherinstance, the first characteristic may be an identity of at least onedrug and the second characteristic may be an identity of another drug.As a result, the clinical outcome provides an indication of aperformance of one drug for treating a patient in comparison to theother drug. In another instance, the first characteristic can be anidentity of a first doctor and the second characteristic an identity ofa second doctor. In this case, the clinical outcome can provide, forexample, an indication of performance of the first doctor in comparisonto the second doctor. The comparison can be related to i) treatment of apatient; ii) treatment of an ailment; iii) use of a drug; and/or iv) thesuccess of a treatment (e.g. surgical technique).

Also in accordance with the invention, the processor instructs the inputmechanism to receive specific sets of medical information based upon anidentity of a user of the digital data processing system. In this case,the clinical algorithm for which all values of the first characteristicare processed is selected based upon the identity of the user of thedigital data processing system. This allows the system to generatedifferent clinical outcome data depending upon who is asking for thedata.

The input mechanism may be coupled to a computer network includingattached geographically diverse patient and doctor computer systems. Inthis case, the user of the digital data processing system can be apatient or doctor who enters at least one of the sets of medicalinformation as input from a patient (or doctor) computer system which isremotely located from the digital data processing system. The outputmechanism thus provides the clinical outcome to the patient (or doctor)over the network immediately after processing the data, therebyproviding immediate feedback in response to entering patient data. Thisallows the data that was input, to be used, for example, in treatment ofa patient at that moment.

In the system, the processor can analyze the clinical outcome forspecific triggering events and can notify a doctor, a patient or othermedical professional upon detection of a specific triggering event thatis determined based on the analysis of the clinical outcome.

Other embodiments of the invention provide certain methods.Specifically, a method is provided for implementing medical studies, andincludes the steps of selecting a medical study and entering medicaldata related to a patient associated with the medical study. Then, themethod immediately processes the medical data entered in combinationwith other data associated with the medical study using a clinicalalgorithm specifically designed for the medical study to produce aclinical outcome of the medical study which takes into account themedical data entered that was related to the patient. The method canalso immediately output the clinical outcome data once processed toprovide an indication as to how the medical data that was enteredeffected the medical study, and can thus be used for immediate helpfulinsight that can be used, for example, to treat a patient or todetermine how a doctor or drug is performing at that point in the studyas compared to other doctors or drugs.

By immediately outputting the clinical outcome data, the method allowsthe clinical outcome data to be used to effect the present treatment ofthe patient based upon the medical data entered.

To select a medical study, the method of the invention includes thesteps of obtaining an identification of an individual and presenting tothe individual a list of medical studies for which that individual isassociated, thereby focusing attention of the individual on particularmedical studies. This saves, for example, a doctors time. The methodalso includes a step of allowing the user to select one of the medicalstudies for which that individual is associated, and, if theidentification of the individual indicates the individual is a doctor,the method can present to the doctor a list of patients associated withthe medical study and allows the doctor to select a patient associatedwith the medical study.

The method can immediately process the medical data entered using aclinical algorithm and can execute the clinical algorithm to producesuch clinical outcome data as i) a comparison of doctors for treatmentof an ailment; ii) a comparison of a drug for treatment of an ailment;iii) a comparison of a physician group for treatment of an ailment; andiv) a comparison of a surgical technique for treatment of an ailment.

To enter the medical data, the method includes the steps of presenting afirst question related to the selected medical study to an individualand retrieving an answer to the question. Then, the method presents asecond question related to the selected medical study to the individual.The second question presented may be determined by the answer retrievedin response to the first question. The method of the invention may thenrepeat the steps of presenting a first question, retrieving an answerand presenting a second question, such that a series of questions arepresented to the individual which are governed by the answers receivedin response to former questions.

The method can also execute the clinical algorithm to determine if themedical data entered does not conform, within a predetermined threshold,to a standardized set of medical data associated with the medical study.If the data does not conform, the method can process a trigger event foran individual associated with the medical study. The trigger event maybe processed for a doctor and the processor, in response to processingthe trigger event, can notify the doctor that a patient has enteredmedical data that does not conform to the predetermined threshold of thestandardized set of medical data associated with the medical study. Thetrigger event may also be processed for a patient. In this case, theprocessor, in response to processing the trigger event, notifies thepatient that the patient has entered medical data that does not conformto the predetermined threshold of the standardized set of medical dataassociated with the medical study and that the patient should seekmedical treatment. The trigger event can also be processed for a medicalprofessional to detect, for example, the requirement for theprescription of a drug.

In accordance with another embodiment of the invention, a method isprovided for performing medical diagnosis. This method receives sets ofcomputerized medical study data and generates comparison resultsdescribing comparisons of the sets of computerized medical study data toproduce a medical study profile (i.e. clinical outcome). Based on themedical study profile, the method provides an indication of a ranking ofa characteristic of the medical study profile. The indication of theranking can provide an indication of any set of computerized medicalstudy data that contains a characteristic that does not conform, withina predetermined threshold, to a standardized characteristic in a typicalset of computerized medical study data, thereby providing an indicationof a need for further medical diagnosis of a patient associated with theset of computerized medical data containing the non-conformingcharacteristic. A trigger event can be generated for a patient and/or adoctor or other entity. The trigger event can notify the patient or thedoctor or the entity of the non-conforming characteristic.

In this manner, feedback can be provided that can be used to effecttreatment of a patient associated with the sets of computerized medicaldata that were received. Certain of the sets of computerized medicaldata may include a set of answers to a set of questions related to aparticular person associated with the medical study. Also, the medicalstudy profile may include a typical set of answers to the set ofquestions. In this case, the step of generating includes, for each setof computerized medical study data, a step of comparing the set ofanswers related to the particular person to the typical set of answersto the set of questions. As such, based upon the comparison of the setof answers to the typical set of answers, the method and system of theinvention can provide a ranking indicative of a deviation of the set ofanswers from the typical set of answers.

As an example, the particular person may be a doctor and the ranking mayindicate a relationship of the performance of the doctor in relation tothe medical study data. The particular person may be a patient as well,and the ranking may indicate a level of treatment provided to thepatient in relation to the medical study data.

Also provided as an embodiment of the invention is a computer programproduct having a computer-readable medium including computer programlogic encoded thereon for determining clinical outcomes of medical data,such that the computer program logic, when executed on a processing unitwith a computing device, causes the processing unit to perform the stepsof the aforementioned methods of the invention. This embodiment isessentially software or code placed on or in a computer disk or memorythat can execute in accordance with the teachings of this invention.This embodiment does not require the clinical outcome system to be acomputer system or to have any hardware, circuitry or other tangibleportion other than the software written in or onto a computer readablemedium. The disk itself with the code that can operate according to theinvention as explained herein is to be considered an embodiment of thisinvention, without a requirement that the code actually be compiled, orif already complied, there is no requirement that it be in execution.

DESCRIPTION OF DRAWINGS

The foregoing and other objects, features and advantages of theinvention will be apparent from the following more particulardescription of preferred embodiments of the invention, as illustrated inthe accompanying drawings in which like reference characters refer tothe same parts throughout the different views. The drawings are notnecessarily to scale, emphasis instead being placed upon illustratingthe principles of the invention.

FIG. 1 illustrates a computer network including a clinical outcomedigital data processing system as configured according to one embodimentof the invention.

FIG. 2 illustrates a more detailed configuration of a data processingsystem configured according to one embodiment of the invention.

FIG. 3 is a high level flow chart of the processing steps performed toprovide clinical outcomes in accordance with one embodiment of theinvention.

FIG. 4 is a flow chart which shows the processing steps performed togather and process data to provide clinical outcomes according to oneembodiment of the invention.

FIG. 5 is a flow chart of the processing steps provided to allow amedical professional to control the general operation of the systemconfigured according to the invention.

FIG. 6 is a data flow diagram indicating various example databases anddata that can be taken into to account when processing a clinicalalgorithm to produce a clinical outcome according to embodiments of theinvention.

FIG. 7 is an example of a detailed view of data that can be generated ina clinical outcomes database from processing clinical outcome algorithmsaccording to embodiments of this invention.

FIG. 8 is a screen shot illustrating a web page interface used to loginto a clinical outcome system configured according to an exampleembodiment of the invention.

FIG. 9 is a screen shot illustrating a web page interface used to selecta study for data entry according to an example embodiment of theinvention.

FIG. 10A is a screen shot illustrating a web page interface used toselect a patient and the next phase of a study for the patient accordingto an example embodiment of the invention.

FIG. 10B is a screen shot illustrating an alternative web page interfacethat can be used to select a patient and the next phase of a study forthe patient according to an example embodiment of the invention.

FIGS. 11A and 11B are screen shots illustrating a web page interfaceused to gather a set of patient medical information according to anexample embodiment of the invention.

FIG. 12 is a screen shot illustrating a web page interface that resultsif patient data is incorrectly or incompletely entered in FIGS. 11A or11B according to an example embodiment of the invention.

FIG. 13 is a screen shot illustrating a web page interface used beginthe review data process according to an example embodiment of theinvention.

FIG. 14 is a screen shot illustrating a web page interface used toselect a study for review according to an example embodiment of theinvention.

FIG. 15 is a screen shot illustrating a web page interface used toselect clinical algorithms and reports that can be generated accordingto an example embodiment of the invention.

FIGS. 16A through 16G are screen shots illustrating web page interfacesproviding various reports and clinical outcome data according to anexample embodiment of the invention.

DETAILED DESCRIPTION

The present invention is directed to techniques providing a tool for theassistance and administration of medical research. FIG. 1 illustrates anexample of a computer networking environment 200 configured to supportoperation of one embodiment of the invention. The computer networkingenvironment 200, in this example, includes a computer network 195 whichinterconnects one or more doctor computers 190, one or more patientcomputers 180, a medical director computer 185, and a clinical outcomesystem 100 which includes a medical database 106 configured inaccordance with the invention. A brief overview of the system of theinvention and its operation will assist in understanding the discussionof more detailed embodiments presented later.

Generally, as illustrated in FIG. 1, the environment 200 allows doctorsand other medical professionals to efficiently carry out medical outcomeresearch according to techniques of the invention. In operation of thesystem 200, doctors (or other medical professionals or medicalresearchers) 187, 188 access doctor computers 190 which interface overthe computer network 195 with the clinical outcome system 100. Clientweb browser software (not specifically shown) executing on the doctorcomputers 190 provides the preferred mechanism for accessing data on theclinical outcome system 100. The clinical outcome system 100 allows thedoctors 187, 188 at the doctor computers 190 and patients 189 at patientcomputers 180 to enter and access patient and medical studies 181 andclinical outcome information 182 maintained within medical databases(i.e., on disks) 106.

As a specific example, the clinical outcome system 100 allows researchdoctor(s) 187 to define a new medical study 181 designed to research andtrack treatment, in real-time, for a particular illness (e.g., breastcancer). The medical study 181, as defined, may include sets ofquestions (not specifically shown in this figure) for gathering medicalinformation from treating doctors 188 and patients 189 who are topartake in research related to the medical study 181. The medicaldatabase 106 stores the medical study 181 and related information (i.e.,clinical outcomes 182) once defined and entered into the clinicaloutcome system 100.

Treating doctors 188 can access the clinical outcome system 100 over thenetwork 195 via doctor computers 190. The various doctor computers 190may be located in geographically diverse locations than the doctors 187and the specific doctor computer 190 used to define the study. Thetreating doctors 188, who may be located throughout the United States,for example, may register to partake in the study 181 if they happen tobe treating a patient 189 that has the particular illness for which thestudy 181 is designed to research. At various times that the treatingdoctors 188 treat the patients 189 having the illness, the treatingdoctors 188 can use their doctor computer 190 to interface with theclinical outcome system 100 for real-time assistance in treatment byusing the study 181.

When accessing the clinical outcome system 100, the clinical outcomesystem 100 presents the treating doctors 188 with a list of studies withwhich they, as participating doctors, are involved. A treating doctor188 can then select the specific study 181 (i.e., breast cancer study)and patient data (not specifically shown) within that study 181 for thepatient 189 to which treatment is about to be administered. The clinicaloutcome system 100 can then present a series of study questions to thetreating doctor 188 at his or her doctor computer 190. The studyquestions presented to the doctor computer 190 may depend upon whatphase of treatment the particular patient 189 is in for the overallcourse or protocol of treatment for the particular illness, as definedin the study 181. In other words, at each phase of treatment for patient189, the study 181 may have particular questions to present to thetreating doctor 188 just prior to a treatment.

Hence, just before treatment is administered, the treating doctor 188can use his or her doctor computer 190 to answer the questions in thestudy 181 as presented by the clinical outcome system 100. The clinicaloutcome system 100 can then process the answers to the questions inreal-time, and can execute specific clinical algorithms (not shown inthis figure) that are related to the study 181 to produce one or moreclinical outcome(s) 182 which are stored in the medical database 106.The answers to the questions are maintained as part of the study 181.The clinical outcome system 100 can then present the clinical outcome(s)182 produced as a result of the doctor 188 providing answers to studyquestions to the treating doctor's 188 doctor computer 190. Based on theresults defined in the clinical outcome 182, which take into account themost up-to-date study data gathered from potentially all patients anddoctors partaking in the study 181, the treating doctor 188 might altertreatment of the particular patient 189 who is about to undergo the nextphase of the protocol or treatment for the illness.

In this manner, the clinical outcome system 100 provides a mechanism tocarry out a medical study 181 and also offers the ability to use themost up-to-date results of the study in real-time to effect treatment atthat moment of a particular patient. By real-time, what is meant is thatthe clinical outcome 182 that is generated can contain the mostup-to-date results which have taken into account the answers provided bythe treating doctor 188 who is seeking the clinical outcome at thatmoment, as well as answers to questions provided by other treatingdoctors and/or patients provided at the same time, or any time prior tothe current time.

As an example related to the above scenario, suppose that the patient189 was entering a phase of breast cancer treatment that provided anoption of obtaining a very new form of radiology treatment. Furthersuppose that the study 181 has been recently defined, edited, created,or modified to take into account the side effects, techniques, benefits,and so forth of the new radiology treatment. Just prior to treatment,the treating doctor 188 can answer questions related to the patient andpatient care at the current phase of treatment. An example would be toprovide information to the clinical outcome system 100 indicating whatdrug treatment has been administered and what the current state of thepatient's well-being is now, just prior to treatment. Based on theanswers to the questions, the doctor 188 (and the patient 189 inattendance at the doctors office or hospital) can obtain clinicaloutcome data 182 that provides up-to-the minute nationwide statisticsconcerning the effects, results, and other characteristics of the newform of radiology treatment and can provide a statistical recommendationand risk stratification, based on the answers to the study questions forpatient 189, as compared to all other study questions and data providedby other patients 189 and doctors 187, 188 around the country up to themoment of processing the clinical algorithm in the clinical outcomesystem 100. This information, provided in “real-time” in response to thetreating doctor 188 answering the study questions, can be used by boththe doctor 188 and the patient 189 when making a decision to proceed ornot with the radiology treatment.

Also as illustrated in FIG. 1, a patient 189 may use a patient computer180 to interact with the clinical outcome system 100. The clinicaloutcome system 100 may have predefined questions in a study 181 that arespecifically designed to be answered, for example, on a periodic basisby the patient 180. The patient computer 180 may be a home-based orpersonal computer located at the patient's 189 residence. This allowsthe patient to contribute to the database 106 containing studyinformation without having to leave the house. This can be particularlyuseful in the administration of drug protocol studies, for example,where a patient 189 must frequently report his or her state ofwell-being and other information in order for the study 181 to beproperly administered and carried-out.

By providing the ability for a patient 189 to enter data to the clinicaloutcome system 100, the invention can also provide clinical outcome data182 back to the patient, in real-time based upon the data entered, toprovide an indication of the patients 189 ranking or aggregate score,based on other patients in the study 181 at the same phase of treatment.For example, based upon the patient data entered, the clinical outcomesystem 100 may provide a patient 189 with clinical outcome data 182 thatindicates he or she is in the top eighty-five percent as an aggregatescore based upon the answers to the questions presented to the clinicaloutcome system 100 for the study 181. In other words, the patient 189can be provided with rather instantaneous feedback of how that patientcompares to others having a similar illness or undergoing similartreatment. This can have a reassuring effect on the patient and avoidsrequiring the patient to have to wait for days or weeks while thepatient data is clinically processed. The clinical outcome system 100can assure the integrity of the study by providing robust errordetection and patient data validation during patient (and doctor) dataentry.

Due to the dynamic nature of the clinical outcome system 100 configuredaccording to the invention, medical databases 106 related to ongoingstudies 181 are continually evolving with the most up-to-date anddetailed data. In other words, the medical database 106 is prospectivein nature and can be constantly collecting sets of medical informationrelated to various studies, patients, and research in a structuredmanner to provide the most up-to-date and accurate clinical outcomes182. The data is collected at a fine level of granularity thus allowingdetails analysis to produce a broad variety of clinical outcomes.

Also as illustrated in FIG. 1, a medical director computer 185 caninterface with the clinical outcome system 100 to obtain particularclinical outcome data 182. In this instance, the medical directorcomputer 185 may be used, for example, by a sponsor of an illness study(e.g., a drug company researching drugs to cure a particular illness),to obtain privileged clinical outcome data 182 that indicates howeffective a particular doctor or group of doctors 187, 188 are attreating the illness under consideration. The clinical data 182 sent tothe medical director computer 185 may be unavailable to patients 189 anddoctors 187 and 188 (at computers 180, 190), based upon access controland security mechanism provided by the system of the invention. Thisfeature of the invention allows, for instance, an HMO to commission andestablish a study 182 to rank doctors 187, 188 or physicians groups(i.e., groups of doctors) on recovery rates of patients 189, as comparedto other doctors and other physicians groups. Based on questionsrequired to be answered in the study 182, and on responses from doctors187, 188, patients 189, and possibly other entities not shown in thefigure, such as hospitals, up to the minute costs can be assessed for aparticular medical treatment. This aspect of the invention may be usedto cross check invoices for medical care, for example, submitted by adoctor 187 or from a physicians group, or from a patient 189. Also, byproviding restricted information, the integrity and privacy rights of apatient 189 may be maintained, while still allowing the patient 189 topartake in the study 182.

Another important aspect of the system of the invention, as conveyed bythe illustration in FIG. 1, is that study data 182 may be used todetermine clinical outcomes 182 that are not possible with prior artsystems since study data 182 is collected at the patient level. Thisallows the system of the invention to provide cross-correlations ofvarious detailed characteristics of sets of medical study 182 data at avery fine level granularity and detail.

By way of example, when a particular patient 189 and doctor 187, 188 areenrolled to partake in a particular study, large amounts of detailedinformation can be initially incorporated into a study 182 withinmedical database 106 for each doctor 187, 188 and patient 189. Eachdoctor 187 can enter information such as the doctors geographicalpractice location, specialty, age, education, past treatments performed,past recovery rates, and so forth into the clinical outcome system 100.Likewise, patients 189 enrolled in the study can initially enterinformation such as age, race, height, weight, build, past medicalhistory, current symptoms, allergic reactions, ages and medicalcharacteristics of parents and relatives, current medications, livingconditions, occupation, salary, siblings and so forth. Since eachpatient is able to enter this information on their own, with little orsometimes no assistance from a doctor 187, 188 or other medicalprofessional, time and money are saved in data entry operations. Thisdata may only need to entered once, at the inception of the study orduring enrollment of a new patient or doctor.

The patient information may be validated upon entry, and can be crosschecked against data that may have been entered, for example, by thatsame patient 189 in a former study. Once the initial bulk study data 181has been entered for a patient 189, during the course of the study, theclinical outcome system 100 can allow obscure queries, via customizedclinical outcome algorithms (not shown in this figure), that candetermine effects and outcome 182 that would have been otherwisedifficult or impossible to detect or determine using prior art studyadministration techniques. This is due to the data being collected, evenfor very large studies, on a patient by patient and doctor by doctorbasis.

For example, if a general clinical outcome 182 indicates a high risk ofcancer for a larger than normal population of young children. Theclinical outcome system 100 allows a more customized clinical algorithmto be quickly developed to gather clinical outcome data 182 related toeach cancer patient under the age of fifteen, and may break the resultsdown by geographical location. Further clinical algorithm analysis maydetermine other statistically common attributes between those children.As a result, patterns of cancer may be detected based upon, for example,certain traits of a child having a specific geographical location, andwhose parents also had a similar type of cancer. Each time a childcancer patient 189 undergoes the next phase of treatment in the study181, the system 100 of the invention can be consulted to determine riskstrata, treatment performance rankings, composite outcomes, and manyother types of information, which are all computed based on up-to-themoment study data.

The results provided by the invention are not possible with prior artsystems since conventional study administration techniques do notcollect the same level of detailed data at the patient level, butrather, gather data as related to groups or a populaces of patients orgather limited sets of information. Moreover, since prior art studymethodologies are not real-time in nature, and cannot evolve as patienttreatments and recovery statistics evolve during the study, prior artstudy techniques are less reliable and the results can quickly becomeout-of-date. The results might even be out of date at the time they arecomputed, due to the non-real-time nature of prior art studyadministration techniques.

Aside from the dynamic nature of the clinical outcome data 182 providedby the invention, the clinical outcome system 100 configured accordingto the invention provides pro-active treatment capabilities in manymedical situations. Specifically, as illustrated in FIG. 1, if a patient189 or a treating doctor 188, during the course of study data entry(i.e., answering study questions at any phase of a study), entersinformation that is of concern or is of an urgent nature, the clinicaloutcome system 100 can causes a trigger event to occur. Essentially, atrigger event, as its name implies, allows the system of the inventionto notify, for example, a medical professional that action is requiredbased on study data that is entered. As an example, if during the courseof required monthly data entry, a patient 189, who is recovering from acancer treatment, enters data that does not conform to a standardized oraccepted level of recovery associated with the current phase of thestudy for the cancer treatment, the clinical outcome system 100 can sendan email notification to the treating doctor 188 for that patient 189.The email may indicate to the treating doctor 188 that the patient 189is not recovering according to the accepted level of progress. As such,the treating doctor 188 can then contact the patient 189 to determinethe cause of the non-conforming recovery. Trigger events, as will beexplained in more detail, can also be established for simply failing toenter data during a certain predetermined window of time during whichthe clinical outcome system 100 expects data to be entered for a patient189 or doctor 187, 188 for a study 181.

As indicated in the above examples, the clinical outcome system 100configured according to the invention allows medical studies to beefficiently carried out. Moreover, the implementation of the system ofthe invention provides significant advantages not found in prior artsystems. Now, a more detailed discussion of certain components andtechniques provided by the invention will be provided.

FIG. 2 illustrates a more detailed architecture of the clinical outcomesystem 100 configured as a digital data processing system according tothe invention. The clinical outcome system 100 is generally a generalpurpose computer or digital data processing system which includes aninterconnection mechanism 101 which couples an input mechanism 102, anoutput mechanism 103, a processor 104, a memory 105, and disks 106 whichstore medical databases (referred to herein generally as medicaldatabases 106) as provided by the invention. A preferred embodiment ofthe clinical outcome system 100 is a high-powered workstation coupledvia a high speed connection to a computer network (e.g., 195 in FIG. 1).Specific examples include a personal computer, Unix workstation,Microsoft Windows NT powered workstation, a cluster of workstations orcomputer, a mini-computer or a mainframe computer.

The input mechanism 102 may be a network connection coupled to computernetwork 195, or may be a keyboard, mouse or other computer input device,or may be itself a standalone computer (e.g., patient computer 180,doctor computer 190). The input mechanism 102 receives sets of medicalinformation, which can include study data, patient data, doctor data,clinical data (answers to questions provided by doctors or medicalprofessionals), drug data, ailment data, treatment data, and so forth.Each set of medical information generally includes characteristicsrelated to a specific medical study and the characteristics of each setgenerally have an associated value.

The input mechanism transfers the sets of medical information, oncereceived, to a storage mechanism, which in this example is the medicaldisk/database 106. The storage mechanism may be a file server, diskdrive, database server (e.g., an Oracle database system manufactured byOracle Corporation), web server or other data storage and retrievalsystem that can store all data and programs related to the invention, aswell as programs, such as an operating system (not specifically shown),which allows the clinical outcome system 100 to operate.

The processor 104 is coupled, via the interconnection mechanism 101, tothe disk drive 106 storage mechanism. As will be explained in moredetail, the processor can process the sets of medical information (e.g.,study data 182 in FIG. 1) according to characteristics by using clinicalalgorithms (not shown) which execute on the processor 104. The clinicalalgorithms produce clinical outcomes 182 for the sets of medicalinformation (e.g., study 181) based upon the selected characteristicsused by the clinical algorithms to process the sets of medical studydata 181.

The output mechanism 103, which may be a network interface coupling aremote patient or doctor computer 180, 190 to the interconnectionmechanism 01 and to the processor 104, receives the clinical outcomes182 generated from processing the sets of medical study information 182.The output mechanism 103 outputs the clinical outcome data 182 to thedoctor or patient users 187 through 189, thus allowing the clinicaloutcome data 182 to be analyzed in real-time, based on the sets ofmedical data 181 received by the input mechanism 102. Also output aretrigger events, which were briefly discussed above and which will bediscussed in more detail later.

The interconnection mechanism 101, may be, for example, a data buswithin the clinical outcome computer system 100 or may be a computernetwork (e.g. 195 in FIG. 1), in which case each of the input and outputmechanisms 102, 103 are separate computer systems (e.g., 180, 190), andin which case the processor 104 is a computer system in and of itselfwhich processes clinical algorithms and stores the clinical outcome data182 in a database in memory and/or disks 105, 106.

The components 101 through 106 of the clinical outcome system 100 of theinvention perform the majority of the processing associated with theinvention, which will be explained next. That is, the processor 104,which is a central processing unit or microprocessor, executes the stepsprovided by the method embodiments of the invention.

FIG. 3 provides a high-level flow chart of the general processing stepsperformed according to embodiments of the invention. In step 130, theclinical outcome system 100 receives medical information as input for aspecific medical study 181. The medical information, as indicated in theabove examples, may be provided to initially establish the study 181, ormay be doctor and/or patient data provided periodically as the study 181progresses. It is to be understood that data for more than one medicalstudy may be input. In step 131, the clinical outcome system 100 storesthe medical information received as input in step 130. The informationmay be stored, for example within a centralized database maintainedwithin disk(s) 106 that are accessible to the processor 104. In step132, the clinical outcome system 100 processes the medical informationaccording to one or more characteristics to determine a clinical outcome182. A clinical algorithm designed specifically for the medicalinformation may be used to process the medical information to producethe clinical outcome. Next, step 133 outputs the clinical outcome datawhich can be used to make medical determinations based upon the datainput in step 130. In the example processing step 133, the clinicaloutcome data is used to effect a patient's case at the time of treatmentand at other times during the study.

An important aspect of the invention is that the clinical algorithm cantake into account medical information that was received in step 131,just prior to processing in step 132. This allows the clinical algorithmto not only generate a clinical outcome for study research purposes(which provides study results using up-to-the-moment collected data),but also allows the clinical outcome to provide the most up-to-datestudy information that is relevant to the person (e.g. doctor 187, 188,patient 189) entering the input in step 130.

The clinical outcome system 100 also takes into account, at the time ofprocessing a clinical algorithm, other medical information that may havebeen input minutes, hours, or only days beforehand from other patients,doctors, medical practitioners, and so forth.

According to this aspect of the invention, the medical information usedby the clinical algorithm may also be obtained from very diversegeographical locations, and thus avoids prior art problems associatedwith having to physically ship medical information to a central locationfor manual processing after treatment has already been undertaken. Sincethe processing is done in real-time, the treatment of a patient, as inthis example, can take into account clinical outcome results from step132. This aspect of the system of the invention was demonstrated in theformer example with respect to FIG. 1, when data reported to a doctor188 and patient 189 from processing a clinical algorithm provided thepatients aggregate performance (i.e., level of treatment or recovery) ascompared to that of the entire group of participants in the study, orsome part thereof. It is particularly valuable that the user (e.g.,doctor and/or patient) can see his or her own aggregate data againstthat of the entire group of study participants (or a subset thereof).This allows, for example, the doctor 188 to compare his practice intreating a specific ailment to a preset subset of other doctors thathave similar practices.

Since the most up-to-date data is used for processing in step 132, andthe doctor is provided with direct access to the system, there is noneed for the doctor to manually request a researching organization toput together a particular clinical outcome based upon manually collecteddata. In other words, the doctors and/or patients themselves, throughthe proper selection of clinical algorithms, can put together or producea particular clinical outcome data set in which they are interested.There can be preset clinical algorithms that can be selected to generatespecific reports for such things as: a percentage of patients enrolledin a study by location; or a doctor/physician practice group versus theentire study group; a composite outcome index which can be derived by acondition-specific algorithm; comorbidity strata by user or user (e.g.,patient) satisfaction data; case rates or medical costs adjusted bycomorbidities for the specific ailment or condition; comparisons ofcomposite outcomes versus treatment protocols or clinical pathways; andso on. It is to be understood that the system of the invention is notlimited to such preset clinical algorithms, and that any particularstudy which is administered by the clinical outcome system 100 can allowcustomized clinical algorithms to be developed based on medicalinformation that is to be collected during the study.

FIG. 4 illustrates a more detailed flow chart of processing steps 150through 161 performed by the clinical outcome system 100 to allow apatient 189 (and/or a doctor 187, 188, as will be explained) to enteranswers to medical study information (enter sets of medical information)for the determination of clinical outcome data according to oneembodiment of the invention. In step 150, a patient 189 logs-in to theclinical outcome system 100. The clinical outcome system 100 providesaccess control functions, user authentication and security in step 150to correctly identify the individual connecting or logging-in to theclinical outcome system 100.

In step 151, based on the identity of the person logging-in, theclinical outcome system 100 presents a list of available studies 181(FIG. 1) to select and the patient 189 (or doctor 187, 199) can select astudy for which the patient 189 or doctor 188 is involved. If a doctor187, 188 is identified as the individual using the clinical outcomesystem 100 in step 151, then the doctor is presented with a list ofstudies, and once a particular study is selected, the doctor 187, 188can then select a patient from a list of patients which that doctor istreating in relation to the selected study.

In step 152, once the clinical outcome system 100 has determined a study181 and patient 189 associated with the study 181, the clinical outcomesystem 100 obtains study questions or a complete study questionnairefrom the study database(s) 106. The particular questions obtained by theclinical outcome system 100 in step 152 may depend upon what phase ofthe study the patient 189 is currently in. In step 153, the clinicaloutcome system 100 presents the questions or the questionnaire to thepatient 189 or doctor 187, 188, in order to obtain a set of medicalinformation in the form of answers to the questions related to theparticular phase of the study 181. Note that the answers provided by thepatient 189 or doctor 187, 188 to certain questions may providereal-time feedback to drive questions presented to the user thereafter.In other words, steps 152 and 153 can be inter-related in such a waythat allows the clinical outcome system 100 to determine from a responseor responses to a question or questions what the next question or groupof questions will be that are presented to the user of the clinicaloutcome system 100.

Note that in step 152 and 153, the clinical outcome system 100incorporates robust data entry validation techniques that help ensurethat the sets of medical information that are entered are accurate.

Once all of the questions have been properly answered, in step 154 theclinical outcome system 100 processes one or more specificcharacteristics of the study data according to one or more clinicalalgorithms associated with the study 181 to produce one or more clinicaloutcomes (e.g. 182 in FIG. 1) that take into account the data entered insteps 152 and 153. The clinical outcome system 100 may obtain theclinical algorithms from a clinical algorithm database, which may bestored along with other medical information in disks 106. More detailsof processing related to the clinical algorithms database and theclinical algorithms and outcomes will be explained later with respect toFIGS. 6 and 7. As previously noted, the clinical outcomes 182 can takeinto account the most up-to-date data entered by the current user (e.g.doctor or patient) as well as data entered by other users participatingin the study 181.

In step 155, the clinical outcome system 100 determines if data existsin the clinical outcome 182 (produced as a result of step 154) that canbe presented to a doctor or other medical professional 187 (doctoroutcome data), 188 or to the patient 189 (patient outcome data). Theclinical outcome data 182 may contain certain data that should only bepresented to a doctor 187, 188, and other data that might be suitablefor viewing by a patient 189.

By way of example, if a patient answers the questions in steps 152 and153, and the clinical algorithm executed in step 154 determines that thepatient is in need of medical treatment (based on responses to questionsin steps 152, 153), then this information might not be suitable to bepresented to the patient 189, for concern of causing unreasonable fear.As such, data of this sort can be filtered by step 155 and presentedonly to a doctor 187, 188 in step 158. Alternatively, if a patient 189is required to routinely (i.e., daily) answer questions in step 152 and153 (from a patient's home computer 180) as required by a drug studybeing administered by the clinical outcome system 100, the clinicalalgorithm processed in step 154 may frequently determine that thepatient answers provided in step 153 conform to normal or standardanswers. As such, the clinical outcome system 100 may detect clinicaloutcome data 182 in step 155 that indicates to the patient 189 that theyare “on-track” for the current phase of treatment in the drug study. Instep 157, the clinical outcome system 100 can send this data to thepatient's 189 home computer 180.

In step 156, the clinical outcome system 100 can analyze the clinicaloutcome data 182 for triggering events that may be associated with thestudy 181. A trigger event, as explained in the former examples, is anevent that may require immediate attention by either the patient 189 ora doctor 187, 188 or other medical professional, and is produced as aresult of analysis of the clinical outcome data 182.

For example, if a patient responds to questions in steps 152 and 153concerning a particular phase of a drug study, and the set of medicalinformation gathered from that patient is analyzed and processed by aclinical algorithm in step 154, the clinical algorithm may determine ormay provide an indication in the clinical outcome data 182 that thepatient requires a second drug to offset a reaction being sustained bythe patient 189. The reaction can be determined to be taking place bythe clinical algorithm based on the patient's 189 responses to the studyquestions in steps 152, 153. As such, the clinical outcome data 182 maysignal a trigger event for the patient's treating doctor 188 to write aprescription to offset the side effect of the drug under study. Thedoctor trigger event detected based on clinical outcome data 182 andthen as generated in step 155 is then processed by the clinical outcomesystem 100 in step 160. The trigger event may for example, cause theclinical outcome system 100 to send the treating doctor 188 an emailnotification indicating a specific drug to be prescribed to a specificpatient 189. The treating doctor 189 can then call the patient 189 toconfirm the diagnosis and can issue the prescription for thecounter-acting drug specified in the clinical outcome data 182 that canbe used to offset the reaction sustained by the patient 189.

Patient trigger events are also provided and processed by the clinicaloutcome system 100 in step 159. Patient trigger events may cause theclinical outcome system 100 to provide an indication to a patient 189(e.g., via email) to call their treating doctor to schedule a visitbased on the clinical algorithm analysis of the responses received insteps 153 and 153. Other trigger events can be processed by the clinicaloutcome system 100 in step 161. An example would be to cause theprescription in the above example to be written and sent to a pharmacyto be filled. Then, when the treating doctor 188 calls the patient 189to confirm the patient reaction, the pharmacy will have already receivedand begun processing the prescription. Another example is to notify asponsor of a study when a particular patient 189 or group of patientshave successfully completed all phases of a study 181.

It is important to understand that the examples given with respect tosteps 150 through 161 in FIG. 4 are merely example uses of the clinicaloutcome system 100 according to the invention. In a situation where theclinical outcome system 100 is used to rate the performance of doctorsagainst other doctors or physicians groups, clinical outcomes mayindicate rankings of doctors by specialty, geographical locations,treatment of particular diseases, and so forth. The trigger events inthese circumstances might be related to signaling the head of aphysicians group, for example, if a clinical outcome 182 of a study 181indicates that a particular doctor 187, 188 is doing excessively poorlyor exceptionally well in treating a particular ailment or patient 189.In the later case, a trigger event may be processed by the clinicaloutcome system 100 in step 161 that provides a compensation bonus to alldoctors in a practice group that meet certain clinical outcome criteria,such as having a treatment success rate combined with a patientsatisfaction rate (as determined by patient input in steps 152, 153) ofover eighty percent. This information can be gained by the system of theinvention in real-time, thus allowing doctor performance in areas oftreatment, patient satisfaction and so forth to be gauged based on themost up-to-date data in the study 181.

FIG. 6 is a flow chart of the processing steps 201 through 213 performedby the clinical outcome system 100 which allow a medical professionalsuch as a doctor 187, 188 or the head of a physicians practice group tointeract with the clinical outcome system 100. In step 201, the medicalprofessional (i.e., non-patient) logs into the clinical outcome system100. Similar identity and authentication checks can be performed, aspreviously explained, for the person logging in to assure the integrityof the clinical outcome system 100. In this description, it is assumedthat the doctor 187 is connecting (i.e. logging in) to the clinicaloutcome system 100.

In step 202, the clinical outcome system 100 presents a list ofselectable options to the doctor 187. In this example embodiment, theoptions include 1) patient data entry, 2) clinical data entry, 3) studyanalysis and 4) new study entry/study edit. Depending upon the users(i.e. doctors) selection, the clinical outcome system 100 processingproceeds to one of steps 203, 204, 207 or 214, as indicated in thefigure. If the doctor 187 selects patient data entry (e.g., a doctorentering answers on behalf of a patient being examined in the doctor'soffice), the clinical outcome system 100 proceeds to step 203, whichdirects processing to step 151 in FIG. 4, to proceed as previouslyexplained. Thus, patient data entry performed by a doctor is essentiallythe same as that performed by a patient, except that, as explainedabove, step 151 allows the doctor to select a study, and then a patientwithin the study.

If in step 202, the doctor 187 selects clinical data entry, then theclinical outcome system 100 proceeds to processing in step 204. In step204, the doctor 187 can select from a number of studies in which thatdoctor 187 may be involved, and can then select a patient in the studyfor which the doctor is currently treating. After study and patientselection, the clinical outcome system 100 proceeds to processing instep 205, where a clinical data questionnaire is presented to the doctor187 for the particular study/patient combination. The clinical dataquestionnaire (not specifically shown in this figure) may contain aseries of questions relating to the medical treatment provided to thispatient in relation to the illness of the patient for the selectedstudy. Thus, the clinical data questionnaire presented by the clinicaloutcome system 100 in step 205 to doctors 187, 188 is generally quitedifferent than the questionnaire presented to patients 189 in step 152of FIG. 4. The clinical data questionnaire is designed to obtain thedoctor's 187 information concerning, for example, how a drug isadministered, how a patient is treated, and how a particular illness orailment is diagnosed by the doctor 187. As in steps 152 and 153 in FIG.4, questions may be presented based upon answers obtained from formerquestions, thus allowing dynamic feedback to drive the sequence ofquestions presented to the doctor 187.

Once the clinical outcome system 100 obtains a complete set of clinicalmedical information for the doctor's 187 responses to the questions instep 205, processing can proceed to step 206, which directs the clinicaloutcome system 100 to process step 154 in FIG. 4. Thus in thisembodiment, when the doctor 187 provides clinical data, step 154 can beused to execute a clinical algorithm to produce clinical outcome data182 which takes into account the doctor's answers provided in step 205.The doctor 187 is able to select the particular clinical algorithmand/or characteristics of study data to be processed in step 154. Thisallows the doctor to obtained customized clinical outcome data 182 foran area of the study data that he or she may have a particular interest.

This is an important aspect of the invention. Since the study data(i.e., sets of medical information collected in steps 152, 153, 205,and, as will be explained, in step 214) is collected on adoctor-by-doctor and patient-by-patient level of granularity, theclinical outcome system 100 of the invention allows very detailedanalysis of study data.

Also, since the study data is maintained in a centralized database ondisks 106, almost any possibly imaginable cross-correlation of data canbe performed by a doctor 187 selecting or defining a clinical algorithmin step 154 that provides the desired research or study results in theform of clinical outcome data 182. The clinical outcome system 100system can even allow cross-correlation of clinical outcome data fromdifferent studies, since the flexibility of the system is limited onlyby the selection of clinical algorithms, which can be customized ordesigned from scratch. Preferably, there are a number of standardclinical algorithms produced when the study is designed, and the doctorswho use the clinical outcome system 100 routinely can select from thestandard clinical algorithms to generate clinical outcome reports.Alternatively, however, the invention is not limited to the pre-definedclinical algorithms.

Returning now to the discussion of processing in step 202 in FIG. 5, ifthe doctor 187 selects study analysis (option 3), the clinical outcomesystem 100 proceeds to process step 207, which allows the doctor 187 toselect a particular study to analyze. Study analysis essentially allowsa doctor 187 or other medical person using the clinical outcome system100 to generate and/or obtain, search and correlate clinical outcomedata, using the clinical algorithms if needed, without having to enterclinical or patient data at that time. The level of study analysisprovided to the user (doctor 187, 188, patient 189) by the clinicaloutcome system 100 in this embodiment is determined by the identity oraccess privileges of the person using the clinical outcome system 100.Thus in step 208, the clinical outcome system 100 in this embodimentdetermines if the medical professional who logged-in in step 201 is adoctor or a medical director. This aspect of the invention allows studyresults to be further filtered according to who is asking for theresults. If an individual doctor is asking, then the clinical outcomesystem 100 can provide certain information, whereas if a medicaldirector sponsoring the study is asking, that person may have fullprivileges to see study data that indicates how all doctors areperforming, for example, in relation to one another.

If a doctor is requesting study analysis in step 208, then the clinicaloutcome system 100 processes step 209 and presents a list of reportsavailable for this particular doctor. A report in this example maysimply be a name given to one or more clinical algorithms to beexecuted. In this instance, the clinical outcome system 100 executeseach clinical algorithm specified in the report to generate clinicaloutcome data in the form of a report on the most recent study data.Alternatively, a report can be selected that may take some time toexecute, and may thus be run in a batch mode. In any event, steps 209and 210 allow a doctor to run reports and clinical algorithms on themost up-to-date study data to determine medical outcomes based upon theclinical outcome data generated as a result of executing each report(e.g., sets of clinical algorithms).

If the clinical outcome system 100 determines that a medical director isrequesting study analysis in step 208, the clinical outcome system 100executes step 211. In step 211, the clinical outcome system 100 allowsthe medical director to select from a list of reports or clinicalalgorithms designed to be executed by medical directors for studyanalysis. Then, in step 212, the clinical outcome system 100 executesany clinical algorithms associated with the selected report (step 211)to generate, for example, performance data on doctors, drugs, treatmentsor ailments. The difference between steps 211 and 212 in comparison to209 and 210 is that a director is presented with reports that cangenerate privileged or sensitive information concerning a study.Information such as doctor rankings, treatment scores, compensationbonuses, treatment and doctor billing rates and costs, and so forthmight be generated when the clinical outcome system 100 executes adirector report. Conversely, a doctor report might generate largeamounts of detailed information that may be of little interest to themedical director of a practice group. Once report data (e.g., clinicaloutcome data 182) is generated, the clinical outcome system 100 outputsthis data from the report or clinical algorithm(s) in step 213.

The processing of steps 208 through 213 allows the system of theinvention to serve multiple goals in relation to the same study. Forinstance, a doctor may be concerned with how his ranking compares withtreatment of an illness versus other doctors, whereas the medicaldirector might be concerned about costs associated with each particulartreatment as compared with the overall performance of those doctors inrelation to one another.

Returning once again to the discussion of step 202, if a medicalprofessional (e.g., a study designer or researching doctor) desires tocreate a new study or to change or edit a study, option four is selectedin step 202 and the clinical outcome system 100 processes step 214. Instep 214, the clinical outcome system 100 allows the user to definestudy parameters, enter and define study questions for patients andclinical questions for doctors, define study timelines, trigger events,specify drug protocols, and so forth. This allows the clinical outcomesystem 100 system to be flexible and adaptable to new studies that arerequired, and allows old studies to be updated with new questions basedon clinical outcome data 182 that can be generated during the life ofthe study.

As an example of this feature of the invention, if a drug protocol studyis underway, and a significant number of patients report having aparticular side-effect from the drug in a general comment questionpresented to the patient in steps 152 and 153, the study designer mayrun a clinical algorithm to determine how often all study patients haveexperienced such side-effect symptoms. If the symptom appearsfrequently, then the study 181 itself may be edited and a question maybe added to the patient and doctor (i.e. clinical) questions list askingall patients if the side-effect is present. This goes far beyond theability of prior art medical study administration systems, which offerno ability to dynamically alter the study once study materials (i.e.,questions) are printed and distributed nationwide.

FIG. 6 illustrates some examples of data that may be incorporated intothe processing of a clinical algorithm 400 according to embodiments ofthis invention. In FIG. 6, the clinical algorithm 400 can access data inany one of databases 401 through 413 to produce the clinical outcomesdatabase 414. Database 401 includes the doctor and patient answers tothe questions presented in steps 152 and 153 in FIG. 4. The expectedresults database 401 includes standard, default or expected answers tostudy questions. Data in the expected results database 402 can becompared with doctor and patient answer data in database 401 todetermine, for example, if a trigger event should be processed in step156.

The available clinical algorithms database 403 can contain the clinicalalgorithms used by studies in the clinical outcome system 100 accordingto the invention. As a clinical algorithm 400 executes, it may call uponother clinical algorithms in database 403. A clinical algorithm itself,may be, for example, a simple or very sophisticated database query.Other databases 404 through 413 that can be accessed by the clinicalalgorithm(s) 400 include information such as drug protocols 404, billingcode determination algorithms 405, study questions and forms data 406,doctor data 407, ailment or illness data 408, physician group data 409,drug interaction data 410, patient medical records data 411, treatmenttechnique data 412, and medical cost data 413. Data from each of thedatabases 401 through 413 can be processed and queried by clinicalalgorithms 400 to produce the clinical outcomes data 414.

FIG. 7 illustrates an example of clinical outcomes data 414. Asillustrated in this example, clinical outcomes data 414 can include suchthings as doctor and physician group performance rankings which includesrankings of doctor and/or group performance with respect to costs,effectiveness of treatments, patient satisfaction, and other factors.Also included in the clinical outcome data 414 can be such things aspatient wellness rankings, suggested treatments and/or procedures forspecific situations, answer information provided by doctors andpatients, clinical data provided by doctors, correlation results of dataanalysis by geographical location, rankings of doctors and patients andphysicians groups by geographical location, drug performance results,treatment performance results, composite outcomes that includecross-correlations of all or selected study characteristics, ailmentrankings, costs basis rankings, recovery and mortality rankings, riskstratification such as by location, and trigger event data. It is to beunderstood that the data illustrated in FIG. 7 is provided as an exampleonly. Clinical algorithms 400 may produce more or less data as required.

One important aspect of the clinical outcome system 100 configured inaccordance with the invention is that the clinical outcomes data 414 iscompletely variable in selection and is produced based on very currentsets of data and medical information. Thus, the clinical outcomes data414 not only can be used as research results for an ongoing study, butcan be used for real-time treatment or instant awareness as to thecurrent state of study data. As explained with respect to the aboveexample, this allows the clinical outcome system 100 to administerstudies and allows doctors to carry out research in a much moreorganized way. Since the doctors are presented with only the study datafor which they are involved, and can then be presented only withpatients in those studies for which they are involved with (i.e.,treating), time is saved for the doctor since his or her attention isfocused on the particular subjects of interest.

FIGS. 8 through 16 shows actual screen shots taken from an example usersession when interacting with a clinical outcome system 100 configuredaccording to the invention and operating generally as previouslyexplained in the former embodiments. The user interface to the clinicaloutcome system 100 as illustrated in FIGS. 8 through 16 is preferablyprovided by world wide web (WWW) browser software, such as NetscapeNavigator, manufactured by Netscape Communications Corporation orInternet Explorer, manufactured by Microsoft Corporation. Preferably,the browser software executes on either a patient computer 180, or, asin this example on a doctors computer 190 (FIG. 1) within the doctor's188 home or office.

In FIG. 8, the clinical outcome system 100 presents the user, which inthis example is a doctor (e.g., 188 in FIG. 1) with a web page display800 that allows the user to provide a username 801 and password 802 foraccess to the clinical outcome system 100. Once the clinical outcomesystem 100 verifies access for the doctor, the clinical outcome system100 presents a screen or web page 810 as shown in FIG. 9 to the doctor188. The web page 810 indicates each study 181 in which the doctor 188is involved, and also provides the identity 812 of the doctor 188. Tabs813 across the top of the web page 810 indicate various functionsoffered by the clinical outcome system 100. In this example scenario,the doctor is about to enter patient data for the lung cancer studynumber 5409 and thus the doctor 188 selects the underlined word “LungCA,” which causes the clinical outcome system 100 to present the webpage 820 in FIG. 10A (or web page 825 in FIG. 10B, as will be explained)to the doctors computer 190.

In web page 820 in FIG. 10A, the clinical outcome system 100 provides alist 821 of all patients 189 (listed by patient numbers) that areinvolved in the study selected from screen 810. For each patient 821,various columns indicate the phase(s) 822 of treatment that each patientin currently undergoing or that they are about to enter. In this examplestudy, there is an initial phase, and then three, six and twelve monthphases 822. For each phase 822 for each patient 821, an “x” 823indicates whether or not that patient 821 has completed that portion orphase 822 of the study. The underlined word “next” 824 indicates a linkto the next uncompleted phase of data entry for the study for eachpatient 821. When the doctor 188 selects (i.e., using a mouse or otherpointing device on the doctor computer 190) “next” 824 in FIG. 10, theclinical outcome system 100 provides the doctors computer 190 with theweb page 830 shown in FIG. 11. As will be explained next, an alternativeinterface to web page 820 is illustrated in FIG. 16H which provides amore detailed indication of phase completion dates and study progressionand remaining study requirements for a particular patient.

FIG. 10B illustrates an example of a web page interface 825 that may beused as an alternative to the web page interface 820 in FIG. 10A. Webpage interface 825 in FIG. 10B may be presented from the clinicaloutcome system 100 to the doctor's computer 190 once a study 181 hasbeen selected from the web page 810 in FIG. 9. Within web page interface825, study phase data (patient name 821, phases 822A through 822D, Surg.Path. 881) is listed for each patient 821 in individual rows. In thisexample, study phase data is illustrated for the patients named “Chris”and “Megan.” Instead of the “Initial,” “3 Month,” “6 Month,” and “12Month” phases 822 as presented in web page 820 in FIG. 10A, web pageinterface 880 in FIG. 10B presents more detailed information concerningthe phases of a study as related to a particular patient 821. The titles883 of each phase, such as “Biopsy”, can be selected, which causes theclinical outcome system 100 to present a form related to treatment orprocedures associated with that phase, such as biopsy treatmentprocedures or patient data collection, as in this example.

In this example related to a breast care study, a study informationsection 882 indicates such things as the name of the study, thephysician ID of the doctor, and a legend 884 to be used to interpret thestudy phase data 822. The legend 884 includes indicators, such as“Italic text,” “Regular Text,” and shading for “Overdue” phases of thestudy. A doctor 188 can use the legend 884 to interpret the study phasedata 882A through 888D, and 881, to determine the detailed studyprogression for each patient 821.

Phase data 822A through 822D can include regular text (i.e., nonitalicized) or italicized labels such as “Initial,” “if performed,”and/or a date, instead of the simple “x” or “next” as used in the webpage 820 in FIG. 10A. Since the clinical outcome system 100 can presentstudy phase data for patients 821 in this manner, doctor(s) 187, 188viewing the web page 825 can gain insight into each patient's 821 statusas related to the study.

For example, for the patient 821 named “Chris”, it is easy for a doctor188 to determine from web page 825 that Chris's six month visit isupcoming on Nov. 11, 1999, since the date “Nov. 3, 1999” is displayed(in italics thus indicating an upcoming phase) for the “6 Month” phase822D of the study phase data for Chris. As another example, it is easyfor a doctor 188 to determine from web page 825 that the “Initial”phases 822A for patients “Chris” and “Megan” have been completed, sincethe clinical outcome system 100 has presented web page 880 with phasedata 822A for these patients that has been shaded with a dark backgroundand the words “Initial” for this phase 822A are indicated in regulartext (i.e., not italicized). Using mechanisms such as shading,italicization, and descriptive titles, the web page interface 825 canprovide very complete information for the current state of a patient's821 progress within a study. This aspect of the system of the inventionsaves considerable amounts of time for a doctor 187, 188 since thedoctor does not have to sort through patient study data to determinewhat the next course of a study is for a particular patient.

FIGS. 11A and 11B illustrates a web page 830 (split between pages due tothe length of data on the web page as displayed within the web browser)that contains patient questions 831 that are to be filled out tocomplete the current phase of the study for the selected patient. Inthis example, there are fifteen questions 831 related to the physicalwell-being of the patient. Since the doctor logged-in in this examplescenario (FIG. 8), the doctor answers the questions on behalf of thepatient who may be present in the doctor's office. The answers 832 inthis example are ranges from one to four and “yes” or “no” indicating astate of well-being for the patient. Once all questions 832 arecomplete, the doctor submits the form using the submit button 833 (FIG.11B).

If form submission is attempted by a user before properly completing allof the questions 832, the web page 840 in FIG. 12 is provided from theclinical outcome system 100 to the doctors computer 190 and indicates841 to the user the missing or required data. Once the form is properlycompleted, form submission using button 833 causes the clinical outcomesystem 100 to present the web page 850 in FIG. 13 which indicates thatthe set of medical information (answers to questions 832) for thepatient has been successfully stored by the clinical outcome system 100.

By selecting the “Review Data” tab 813-1 in FIG. 13, the clinicaloutcome system 100 presents the web page 860 in FIG. 14. The doctor maythen select one of the studies 181 to analyze clinical outcome data. Inthis example scenario, the doctor selects the “Lung CA” study 181,having study ID number 5409 and the clinical outcome system 100 presentsthe web page 870 in FIG. 15.

The results page 870 indicates a number of clinical algorithms orreports 871 that may be selected. When the doctor selects a specificclinical algorithm 871, the clinical outcome system 100 generatesmedical outcome data (182 in FIG. 1), in real-time, by analyzing themost current sets of medical information entered for the particularstudy according to the queries and instructions defined for theparticular selected clinical algorithm 871. In this example, the dataentered for questions 832 is taken into account during this analysis.

FIGS. 16A through 16F illustrate various example clinical outcome datasets or reports generated by the clinical outcome system 100 when a userselects various clinical algorithm reports 871 from the web page 870 inFIG. 15. Example reports include: patients entered by site (FIG. 16A);composite outcomes for pre- and post-treatment for the study (FIG. 16B);stage-comorbidity strata for a site and all patient participants in thestudy (FIG. 16C); Quality of Life using SF-36 (a standard measuringtechnique) (FIG. 16D); Patient satisfaction survey (FIG. 16E), Cost rateper case by strata and site (FIG. 16F); and Risk adjusted compositeoutcome versus clinical pathway in a specific year of the study (FIG.16G).

The example clinical algorithms producing outcomes, can for example,provide risk correlations that can be adjusted for baseline risk factorsfor patients. For example, if a study is used to monitor cancertreatment results, and some patients were smokers while others did notsmoke, the correlation may weigh the non-smoker's results more heavilythan the smoker's, due to the fact that the person who smokes has ahigher risk from dying from a smoking related illness such as heartdisease. Such correlations are possible in the invention since the datais collected patient-by-patient and details such as these are availableon a widespread basis.

It is to be understood that these reports are merely examples ofpossible reports and queries that can generated by clinical algorithmsproduced for the clinical outcome system 100 of this invention. Thereport data shown in FIGS. 16A through 16G may be saved in the clinicaloutcome database 414 for use in future research, and can provide asnapshot of the progress of the study thus far.

Since the system of the invention takes into account the most up-to-datemedical information to generate each report, as the study evolves, theresults of each clinical algorithm report may change as well. Moreover,the clinical algorithms themselves may change to take into accountthings discovered during the study which should be factored in to theprocessing of clinical algorithms. Clinical algorithms and studies maybe edited, for example, in step 214 in FIG. 5, as explained previously.At the end of a study, the reports generated over time for particularclinical algorithms may be further analyzed to provide a progressivetimeline of study development, and may prove useful in and of themselvesas research data.

While this invention has been particularly shown and described withreferences to preferred embodiments thereof, it will be understood bythose skilled in the art that various changes in form and details may bemade therein without departing from the spirit and scope of theinvention as defined by the appended claims.

1. A digital data processing system, comprising: memory to storeinstructions for execution; one or more processing devices configured toexecute the instructions, the instructions for causing the one or moreprocessing devices to: receive medical data associated with a medicalstudy into a memory of a computer system, the medical data includinginformation in accordance with definitional information for a medicalstudy; process the medical data using a clinical algorithm executed byat least a processor of the computer system to determine first clinicaloutcome data, the first clinical outcome data indicating aneffectiveness of treating patients associated with the medical study;output at least a portion of the first clinical outcome data to a user;receive input from the user during the course of the medical study;generate modified definitional information for the medical study basedon the input; and repeat the receiving and processing of additionalmedical data using the modified definitional information for the medicalstudy during the course of the medical study to determine secondclinical outcome data.
 2. The digital data processing system of claim 1,wherein receiving the medical data, processing the medical data, andgenerating the modified medical study definitional information occur insubstantially real-time during which the medical data is gathered in anongoing manner during the medical study as the medical study evolvesover time based on the input.
 3. The digital data processing system ofclaim 1, wherein outputting at least a portion of the first clinicaloutcome data comprises: outputting at least a portion of the firstclinical outcome data to a user according to a user characteristic ofthe user.
 4. The digital data processing system of claim 3, wherein theuser characteristic comprises at least one of an identity or a privilegelevel of the user.
 5. The digital data processing system of claim 1,wherein the instructions further comprise instructions for causing theone or more processing devices to: compare the medical studydefinitional information with the modified medical study definitionalinformation to determine an extent of development of the medical study.6. The digital data processing system of claim 1, wherein the inputreceived from the user comprises an instruction to redefine an elementof the medical study, wherein the element comprises at least one of amedical study parameter, a question to be asked in the medical study, atimeline of the medical study, a trigger event, or a drug protocol. 7.The digital data processing system of claim 6, wherein generating themodified medical study definitional information comprises: changing atype of medical data to be gathered in accordance with the medical studydefinitional information during a subsequent portion of the medicalstudy.
 8. The digital data processing system of claim 6, whereingenerating the modified medical study definitional informationcomprises: modifying a list of questions associated with the medicalstudy, the list of questions being for presentation to at least one ofpatients of the medical study to gather the additional medical data orthe doctors to gather the additional medical data on behalf of thepatients.
 9. A computer program product having a computer readablemedium comprising computer program logic encoded thereon, such that thecomputer program logic, when executed on at least one processing unitwith a computing device, causes the at least one processing unit toperform the steps of: receiving medical data associated with a medicalstudy into a memory of a computer system, the medical data includinginformation in accordance with definitional information for a medicalstudy; processing the medical data using a clinical algorithm executedby at least a processor of the computer system to determine firstclinical outcome data, the first clinical outcome data indicating aneffectiveness of treating patients associated with the medical study;outputting at least a portion of the first clinical outcome data to auser; receiving input from the user during the course of the medicalstudy; generating modified definitional information for the medicalstudy based on the input; and repeating the receiving and processing ofadditional medical data using the modified definitional information forthe medical study during the course of the medical study to determinesecond clinical outcome data.
 10. The computer program product of claim9, wherein receiving the medical data, processing the medical data, andgenerating the modified medical study definitional information occur insubstantially real-time during which the medical data is gathered in anongoing manner during the medical study as the medical study evolvesover time based on the input.
 11. The computer program product of claim9, wherein outputting at least a portion of the first clinical outcomedata comprises: outputting at least a portion of the first clinicaloutcome data to a user according to a user characteristic of the user.12. The computer program product of claim 11, wherein the usercharacteristic comprises at least one of an identity or a privilegelevel of the user.
 13. The computer program product of claim 9, whereinthe computer program logic, when executed on the at least one processingunit with the computing device, causes the at least one processing unitto perform the step of: comparing the medical study definitionalinformation with the modified medical study definitional information todetermine an extent of development of the medical study.
 14. Thecomputer program product of claim 9, wherein the input received from theuser comprises an instruction to redefine an element of the medicalstudy, wherein the element comprises at least one of a medical studyparameter, a question to be asked in the medical study, a timeline ofthe medical study, a trigger event, or a drug protocol.
 15. The computerprogram product of claim 14, wherein generating the modified medicalstudy definitional information comprises: changing a type of medicaldata to be gathered in accordance with the medical study definitionalinformation during a subsequent portion of the medical study.
 16. Thecomputer program product of claim 14, wherein generating the modifiedmedical study definitional information comprises: modifying a list ofquestions associated with the medical study, the list of questions beingfor presentation to at least one of patients of the medical study togather the additional medical data or the doctors to gather theadditional medical data on behalf of the patients.
 17. Acomputer-implemented method comprising: receiving medical dataassociated with a medical study into a memory of a computer system, themedical data including information in accordance with definitionalinformation for a medical study; processing the medical data using aclinical algorithm executed by at least a processor of the computersystem to determine first clinical outcome data, the first clinicaloutcome data indicating an effectiveness of treating patients associatedwith the medical study; outputting at least a portion of the firstclinical outcome data to a user; receiving input from the user duringthe course of the medical study; generating modified definitionalinformation for the medical study based on the input; and repeating thereceiving and processing of additional medical data using the modifieddefinitional information for the medical study during the course of themedical study to determine second clinical outcome data.
 18. Thecomputer-implemented method of claim 17, wherein receiving the medicaldata, processing the medical data, and generating the modified medicalstudy definitional information occur in substantially real-time duringwhich the medical data is gathered in an ongoing manner during themedical study as the medical study evolves over time based on the input.19. The computer-implemented method of claim 17, wherein outputting atleast a portion of the first clinical outcome data comprises: outputtingat least a portion of the first clinical outcome data to a useraccording to a user characteristic of the user.
 20. Thecomputer-implemented method of claim 19, wherein the user characteristiccomprises at least one of an identity or a privilege level of the user.21. The computer-implemented method of claim 17, further comprising:comparing the medical study definitional information with the modifiedmedical study definitional information to determine an extent ofdevelopment of the medical study.
 22. The computer-implemented method ofclaim 17, wherein the input received from the user comprises aninstruction to redefine an element of the medical study, wherein theelement comprises at least one of a medical study parameter, a questionto be asked in the medical study, a timeline of the medical study, atrigger event, or a drug protocol.
 23. The computer-implemented methodof claim 22, wherein generating the modified medical study definitionalinformation comprises: changing a type of medical data to be gathered inaccordance with the medical study definitional information during asubsequent portion of the medical study.
 24. The computer-implementedmethod of claim 22, wherein generating the modified medical studydefinitional information comprises: modifying a list of questionsassociated with the medical study, the list of questions being forpresentation to at least one of patients of the medical study to gatherthe additional medical data or the doctors to gather the additionalmedical data on behalf of the patients.